One Body, Many Minds: Exploring Design Frictions and Considerations for Multi-Agentic Systems in Human-Robot Interaction

Abstract

When multi-agentic systems are implemented in human-robot interaction, users interact with a single physical robot while multiple AI agents operate ’behind the curtain’. This creates a fundamental mismatch: the system’s multi-agent nature is hidden from users, but understanding agent roles, capabilities, and orchestration is critical for system transparency and user understanding. We investigate how multi-agentic systems can be represented through a single anthropomorphic robot. We conducted a design workshop with seven HCI experts to explore design parameters for system characteristics (e.g. objective, persona, tool, handoff) and discuss orchestration configurations through design probes de..